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Summary. We derive a high-resolution formula for the quantization problem under 
Orlicz norm distortion. In this setting, the optimal point density solves a variational 
problem which comprises a function g : R_|_ — >■ [0, oo) characterizing the quantization 
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we find convergence as in the classical setting. 
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1 Introduction 

For d € N, consider an M*^- valued random vector X {the original) defined on some proba- 
bility space {Q,A,F), and denote hy /j, = C{X) the law of X. 

We consider the quantization problem, that is for a given natural number € N and 
a loss function p : M*^ x M'^ ^ [0, oo) we ask for a codebook C C M'* consisting of at most 
N elements and minimizing the average loss 

Ep{X,C), 
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where p{x,A) = infy^zA p{x,y), for all x E M*^ and A C W^. The quantization problem 
arises naturally when discretizing analog signals, and it first gained practical importance 
in the context of pulse-code-modulation. Research on it started in the 1940's and one finds 
numerous articles dedicated to the study of this proble m in the engineering liter ature. For 
an overview on the s e dev elop ments, one may consult iGrav and Neuhoff (|l998l ^ (see also 
Cover and ThomasI (|l99ll ) and lGersho and Gravl (119921)'). The quant ization problem is also 
related to numerical integration Pages. Pham and Printemsl ( 20041 ) . and, more recently, 
the mathematical community became attracted by the field. In the last years a number 
of new publications ap peared treating fini t e dirn e nsional a s wel l as infinite dimensional 
sign als (see for instance Graf and Luschgy ( 200d). Gruber (2004) for vector quantization 
and Luschgv and Pagfei ( 2004 ) , Dereich et al. ( 2003 ) for functional quantization) . 

In this article, we consider asymptotic properties of the quantization problem when 
the size N of the codebook tends to infinity, the high-resolutio n quantization pro blem. 
First asymptotic formula e for vector quantization were found by IZadoii (jl966l . Il982l ) and 
Bucklew and Wise! (|l982l ). 

In the classical setting (norm based distortion), one considers 



p{x,y) 



\x - y\ 



for some norm || • || on and a moment p ^ 1. As long as the distribution // := C{X) has 
thin tails (in an appropriate sense) one can describe asymptotically optimal codebooks via 
an optimal point density function: the empirical measures associated to optimal codebooks 
C{N) of size 

oteC 

converge to a continuous probability measure that has density proportional to {^^)'^^^'^~^^^ ■ 
The density will be called optimal point density. Here and elsewhere, A'^ denotes d- 
dimensional Lebesgue measure and pc denotes the absolutely continuous part of fi w.r.t. A*^. 
When considering point densities we will always assume that pc does not vanish. Optimal 
codebooks for the uniform distribution can then be used to define asymptotically optimal 
codebooks for general X: roughly speaking, one partitions the space M.'^ into appropriate 
cubes and chooses in each cube an optimal codebook for the uniform distribution of ap- 
propriate size (according to the optimal point density). In particular, the non-continuous 
part of fi has no effect on the asymptotic problem. The concept of a point density will 
play a crucial role in the following discussi on. Its i i nport ance in the classical setting was 
fir st conjectured b y Lloyd and Gersho (see Gersho ( 19791 )). First rigorous proofs are due 
to 



Bucklewl (|l984l ). For a recent account on the theory of hi gh resolution quant i zation and 



point density functions one may consult the monograph by Graf and Luschgv ( 2000l ). 

Nowadays, the asymptotic quantization problem is well understood for loss functions 
p that are shift invariant and look locally like a power of a norm based distance, that is 

p{x, y) = p{x — y) and p{x) = + o(||xP) as x — )• 



for some norm || • || on M and a power p ^ 1 (see belattre et (l2004l '». Here and 
thereafter o and O denote the Landau symbols. For all these distortion measures one 
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regains the same optimal point density as for the corresponding norm based distortion 
measures. Since the optimal point density only depends on the behavior of p close to 0, 
quantization schemes based on this density may show bad performance for moderate N . 
This will occur, when for the original X and the approximation X the distances p{X, X) 
and ||X — X|p differ significantly. 

As a generalization of the above setting, we suggest to use Orlicz norms as a measure for 
the loss inferred when approximating the original X by the closest point in a codebook C. 
One reason for this is that optimal codebooks for general distortions are also optimal 
codebooks for certain Orlicz norm distortions. For moderate N the optimal point density 
of the corresponding Orlicz norm distortion seems to be the favorable choice as basis for 
the construction of good codebooks since it incorporates p as a whole and not only its 
asymptotics in zero. 

Let us introduce the main notation. Let (p : [0, oo) — ^ [0, oo) be an increasing, left 
continuous function with lim^io v(^) = 0- Note that this implies that is lower semicon- 
tinuous. We assume that f ^ Q,lei E = iW^, \\ ■ ||) denote an arbitrary Banach space, and 
denote by d{-, •) the associated distance. For any R'^-valued r.v. Z, the Orlicz norm || • \\^p 
is defined as 

\\Z\\^ = infjt ^ : E<^(^^ ^ 

with the convention that the infimum of the empty set is equal to infinity. Actually, the left 
continuity of ip together with monotone convergence imply that the infimum is attained, 
whenever the set is nonempty. We set 

L'^(P) = {Z :Z M'^-valued r.v. with \\Z\\^ < oo}. 

Note that || • ||<^ defines a norm on L'^{F) when ip is convex, whereas otherwise, the 
triangle inequality does not hold. For our analysis we do not require that (p be convex. 
Nevertheless, with slight misuse of notation, we will allow ourselves to call || • 11,^ an Orlicz 
norm. Choosing (p{t) = t^, p ^ 1, yields the usual i7'(P)-norm, which will be denoted by 
II ■ lip- 

For iV ^ 1, we consider the quantization error given by 

5{N\X,p) =inf 

X 

where the infimum is taken over all r.v.'s X (reconstructions) satisfying the range con- 
straint jrange {X)\ < N. In the case where ip{t) = tP for ap > 0, we write briefly S{N\X,p) 
for the corresponding quantization error. 

Let us compare the Orlicz norm distortion with the classical setting. Suppose that X 
is an optimal A^-point quantizer under the distortion p{x,y) = f{\\x — y\\), where / is a 
left continuous and strictly increasing function with /(O) = lim^^o f{t) = 0. Then one can 
easily verify that X is also an optimal A^-point quantizer in the Orlicz norm setting when 
choosing p{t) = f{t)/A and A = E[p(A, A)]. Thus optimal quantizers in the classical 
setting correspond to optimal quantizers in the Orlicz-norm setting. As we will see later, 
in most cases each choice of A (or (p) leads to a unique optimal point density function. 
Thus the optimal point density function in the classical setting is replaced by a whole 
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family of densities: good descriptions are now obtained by choosing the parameter A 
accordingly. We believe that the optimal Orlicz point density is a favourable description 
of good codebooks for moderate A^. Let us illustrate this in the case where f{t) = ip{t) = 
exp(t) — 1 (meaning that A = 1). Whereas the Orlicz norm point density attains rather 
large values even at points where the density is very small (see Example II. 4p . the 
classical setting neglects the growth of / and one retrieves the optimal density of the 
norm based distortion to the power 1. 

In our notation, the asyrn ptotics under the classical L^-norm distortion reads as follows 
(see Graf and Luschgv ( 200d )): Let p ^ 1, U denote a uniformly distributed r.v. on [0, l)*^, 
and set 

q{E,p) = mf^N^/'^5{N\U,p). 
If for some p > p, M\\X\\p < oo (concentration assumption), then 



lim N'^/'^6{N\X,p) =q{E,p) 

N^oo 



i/p 

(1) 

i^d/{d+p)i 



where iic denotes the absolutely continuous part of /i. 

The analysis of the Orlicz norm setting is based on the concept of a point allocation 
density. We shall see that an optimal point allocation density is given as a minimizer of 
a variational problem. Unfortunately, in this context the minimization problem cannot 
be solved in closed form. We will prove the existence and a dual characterization of the 
solution. The quantity q{E,p) corresponds in the general setting to a convex decreasing 
function g : (0, oo) — )• [0, oo) which may be defined via 

5(r/)= hm inf E^{{N/rj)'/''d{U,C{N))), (2) 
N^ooC(N) 

where the infima are taken over all finite sets C{N) C R'^ with |C(iV)| ^ N and U de- 
notes a uniformly distributed r.v. on the unit cube [0, 1)"^. Moreover, we set g{0) = 
liminf^io 5(^) ^ [0,oo]. The function g depends on the Banach space and the Orlicz 
norm. It will be analyzed in Section [2l g can be represented as a particular integral in 
the situations where E = l'^ ox E = If 9? induces the LP(P)-norm (i.e. ip{t) = t^), then 
due to 

g{r^)=q{E,pYr,-P/''. 

If the measure ^ is not compactly supported, our analysis relies on a concentration 
property. As in the L*'(P)-setting, this can be done by assuming the finiteness of an 
integral E^dlXH) for some function ^ satisfying a growth condition (Condition (G), see 
Definition 17. 2p . Let us state the main theorem. 

Theorem 1.1. Assume that ^ satisfies the growth condition (G) and that'&^{\\X\\) < 00. 
Then 

lim N^^'^6{N\X,^) = I^/'^, 

N^oo 

where I is the finite minimal value in the point allocation problem. It is given by 

/ = inf j ^{x)dx, (3) 
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where the infimum is taken over all non-negative Lebesgue integrable functions with 

g{C{x)) dnc{x) ^ 1. 

F 

Alternatively, one can represent I by the dual formula 

/ = sup-(/ g{--^) dfic{x) - l), 

where g{t) = inf^>o[5(??) + r]t] and h{x) = ^^l^{x)- Moreover, one has I > if and only if 

//e(K°') sup ifit) > 1, 
t ^ 

where we use the convention that • oo = 0. 

Remark 1.2. • If we choose (p{t) = t^ in the former theorem, we obtain the classical 
result, since ^{t) = with q > p satisfies the growth condition (G), see Example 17.31 
for an even weaker assumption. 

• The point allocation problem is studied in Section [5l In particular, representations 
for optimizers can be found in Theorem 15.11 and Remark 15.21 

We keep I as the minimal value in the point allocation problem given by ([2]). If / is 
strictly bigger than 0, we denote by Ai the set of probability measures on the Borel sets 
of M.'^ associated to the minimizers of the point allocation problem, i.e.. 



e, / g{I^{x))dfi,{x) = l, [ a^)dx = l}. (4) 



Theorem 1.3. Assume that I G (0, oo) and denote by C{N), € N, asymptotically 
optimal codebooks of size N, that is 



limsupiVi/'^||d(X,C(A^))||^ ^ I^/'^. 



Then the empirical measures vn given by 



N 



x€C(N) 



form a tight sequence of probability measures, and any accumulation point o/(i/Ar)iVGN H^s 
in AA. If g is strictly convex, then the set A4 contains exactly one measure v, and 

lim i/jv = weakly. 

Af-s>oo 

As an example we present implications of our results for the standard normal distri- 
bution under a particular ip growing exponentially fast: 
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Example 1.4. Let ^ denote the standard normal distribution, {E, \\ ■ ||) = (M, | • |) and 
(p : [0, oo) [0, oo), X 1-^ exp(x) — 1. Then ^(x) = exp(x^/^) satisfies the growth condition 
(G) (see Example 17. 3p . and Theorem 11.11 is thus applicable. Moreover, (see Remark 12.21 
and Lemma |2.3|) 

1-1/2 

g(r]) = 2 ip{t/r]) dt = r/e^/^^'') - 2rj - 1. 







Note that g ist strictly convex, so that there has to exist a unique normalized optimal point 
density ^. In order to approximate the optimal value for k, we used numerical methods 
to obtain n « 0.699. Due to and the optimal point density is given by 



e(x) = (-5'r(x^), 

and / = J^^ix) dx k. 2.88 . Next, elementary calculus gives 

( — q')~^it) = -- — r- ; — — r as t — > oo 

^ ^' 21og(t)-log(21ogt) + o(l) 

so that the normalized point density i = 5,/! satisfies 



u ^ 11 II 
c,[x) ~ -—^ as \x\ oo. 

1 x^ 



Hence, decays to zero much more slowly than in the classical setting. 

The article is outlined as follows. In Section^ we begin with an analysis of the function 
g. In Section [3] we construct asymptotically good codebooks based on a given point 
allocation measure. Up to this stage, we are restricting ourselves to absolutely continuous 
measures with compact support. In Section U we turn things around and prove a lower 
bound based on a given point density measure. This bound implies the lower bound in 
Theorem 11.11 and proves a part of Theorem 11.31 The estimates of Sections [3] and H] lead 
to the variational problem characterising the point density, and this is treated in Section 
[5l In the last two sections, we treat the upper bounds in the quantization problem for 
singular and non-compactly supported measures. In particular, we derive a concentration 
analog which guarantees that the quantization error is of order 0{N-^l'^). Finally, we 
combine the estimates and prove the general upper bound in Theorem 11.11 

It is convenient to use the symbols ^, < and ~. We write / ~ g iff lim ^ = 1, while 

f ^ g stands for lim sup ^ ^ 1- Finally, f ~ g means < liminf ^ ^ lim sup ^ < oo . 

2 First estimates for the uniform distribution 

In this section, X denotes a uniformly distributed r.v. on [0, l)"'. For T? > and iV ^ 1, 
we consider 

/^(r?) := Jnf^Ev9((iV/r?)V'^ _min ^ ||X - x||) , (5) 
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where the infimum is taken over all codebooks C{N) C M"^ of size [A^J . Here and elsewhere, 
[A^J denotes the largest integer smaller or equal to A^. By a straightforward argument, 
the lower semicontinuity of ip implies that the function 

(M'^)L^J 9 ^E(^f(iV/r/)i/'^ min \\X - Xi\\) e [0,oo) 

V i=l,...,[N] / 

is also lower semicontinuous. In the minimization problem ([5]), it suffices to allow for 
codebook entries that are elements of a sufficiently large compact set. So the lower semi- 
continuity implies the existence of an optimal codebook. We usually denote by X^^^ or 
Xi^^v) an optimal reconstruction attaining at most different values, that is X = X^^^ 
is a minimizer of 

E^(^{N/i])^/'^ ||a: 

among all r.v.'s satisfying the range constraint |range {X)\ ^ A'^. Now define the function 
9 by 

We start with a derivation of the structural properties of g. In particular, we show the 
vahdity of Q. 

Theorem 2.1. The function g : IR+ — )• [0, cxd) is decreasing and convex, and satisfies 
lim^^oo gir]) = and lim^4,o g{r]) = sup^ ^ q ip{t). Moreover, for r] > 0, 

lim Eip( (N/rifl'^ \\X - X^^M\ \ = gir]). (6) 

We will sometimes use the convention 5^(0) = lim^|o5'(^)- Note that ^(0) is finite iff (p 
is bounded. Moreover, we set fNi'n) = oo for A^ € [0, 1). 

Remark 2.2. In general, computing the function g explicitly constitutes a hard problem. 
However, as for the classical L'^-norm distortion, one can calculate g when = M*^ is 
endowed with supremum-norm, and in the case where E is the two dimensional Euclidean 
space. In such cases, the same lattice quantizers can be used to construct asymptotically 
optimal codebooks and to compute the function g. The case of the supremum-norm is 
trivial since the unit ball is space filling. 

Lemma 2.3. Let U be uniformly distributed on a centered regular hexagon V in M? having 
unit area, and assume that E is the 2-dimensional Euclidean space. One has 



a 



The proof is similar as in the classical set ting and therefore ommitted, see (iGraf and Luschgv . 

2nnnl . Theorem 8.15) and lPeies T6thl (|l972l ). 



In order to prove Theorem 12. H we use the inequality below. It is essentially a conse- 
quence of the self similarity of X. 
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Proposition 2.4. Let M € {k"^ : k G N}, ?/2 > > and let rj = arji + /3?72 be a convex 
combination of tji and r/2 • Then for any N ^ 1 one has 

fNiv) ^ a{M) fN./Mim) + h{M) fN./uim), (7) 

where 

. Ni = ^N and = ^N, 

• b{M) = [(3M\/M and a{M) = 1 - b{M). 
Additionally, one has for N ^ 1 and r] > 0, 

fNiv) ^ fN/Kliv)- (8) 

Proof. Fix € N and let 771, 772, a, /3, iVi, iV2, M = k"^ be as in the proposition. Let 
Ci = [0, We decompose the cube [0, l)'^ into an appropriate union [J^LiCi of 

disjoint sets Ci, . . . , Cm, where each set C2, . . . , Cm is a translate of Ci. Moreover, let Xi 
denote a uniformly distrib uted r.v. on d. S i nce U {\Q, 1)'^) = jj YlJiLi^iCi)-, one has, in 
analogy to Lemma 4.14 in iGraf and Luschgvl jioO^), 



1 ^ 

fMiv) =eMn/v)"' \\x - ^ i^Y^M^^I^)'^' 11^^ - 



M 

M 

i=l 

for any [1, cxd)- valued sequence {Ni)i=i,...,M with ^ ^- Here, X^^^^ denotes an 

optimal quantizer for Xi among all quantizers attaining at most Ni different values. Since 
the distributions U{Ci) can be transformed into lA{Ci) through a translation, one obtains 

/^(^) ^j^Y.^^{{N/v)"''\\Xi-X^^^\ 

i=l 



Self similarity {C{Xi) = C{tX)) then implies that 



1 / 1 



i=l 



Note that the assertion of the proposition is trivial if Ni/M < 1, so that we may assume 
N2/M Ni/M 1. We now choose for [/3MJ indices iVj = N2/M and for M - [/3MJ 
indices Ni = Ni/M. Then Y^- Ni ^ iV, so that 

fNiv) ^ a(M)E(^((iV/7?)i/'^i ||X 

+ b{M) Eipi^iN/v)^^"^ i ||X - X(^2/Af) 
= a(M) /^,/m(??i) + b{M) fN,/M{V2), 
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where b{M) = [/3MJ/M and a(M) = 1 - b{M). Analogously, setting Ni = N/M for 
i = 1, . . . , M, we obtain that fNiv) ^ fN/Miv)- ^ 

Proof of Theorem 12.11 Obviously g is decreasing. First we prove that for arbitrary 
r/ > 0, 

g{ri) ^ limsup/Ar(T/) ^ g-{r{). (9) 

Fix e > and choose r/o G (0,?/) so that ^(r/o) ^ d-i'l) + £/2- Moreover, fix A'^o ^ 1 
with fNoiv) ^ fi'l^) + For ^ A'o, we decompose A" into N = Nok'^ + N, where 
k = k{N) € N and iV = N{N) € No are chosen so that N < {k + l)'^A'o. Then 

/;v(r?)=E(^((A^/7?)i/'^||A-AW||) 

^ E (^((A^o k'^/iNo kS/N)) ^1'' II A - A(^° ll) = (r/A^o k'' jN), 

and inequality ([H]) implies that for M = M{N) = k'^: 

fN{v)^fNo{vNoM/N). 

Note that Aq^"' ^ A^ < Afo(A; + 1)'^, hence: limAr^oo ^/A^o^"'/A^ = V- Consequently, there 
exists A^i ^ A^o such that for all N ^ Ni one has: t^NqM/N ^ 779, and 

fNiv) ^ fNoivNoM/N) ^ fNoivo) ^ g-{ri)+e 

for all N ^ Ni. Since e > was arbitrary statement ^ follows. 

We now prove that g- is convex. Let f?2 > f?i > and let rj = arji + /3t/2 be a convex 
combination of 771 and rj2 and suppose that g-{rji) is finite. Fix A; G N and let M = k'^, 
a{M) and b{M) be as in Proposition 12.41 Moreover, for given A^ E N we let A^i = Ni{N) 
and A^2 = N2{N) be as in the previous proposition. Then inequality ^ implies that 

fN{r]) ^ a{M) fN,/M{m) + KM) fN^/uim)- 

Therefore, formula Q and the left continuity of g- give 

g{il) ^ limsup/^(r?) ^ a{M) g^{r^i) + b{M) g^{l^2). 

Recall that M G {k'^ : A; € N} was arbitrary. Since limM->cxD a{M) = a and limjv/^00 b{M) = 
j3 we conclude that 

For the general statement, observe that 

g~{v) = ^'oigiv -S) ^ lim sup [a 5_ - 6) + /S g^{rj2 - 6)] = ag_(??i) + /3fif_(r/2). 

Consequently, g- is convex, and a forteriori it is continuous. Therefore, the functions g 
and g- coincide, which proves ([6]). 
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It remains to prove the asymptotic statements for g. First note that 

giv) ^ fM ^ E99(r?-i/'^ ^ ifiT^-^/^ sup 

a:6[0,l)d 

as 7/ — >■ GO. On the other hand, one has for > 0, e > and AT G N, 
/jv(??) = E(^((iV/77)V<^||X-XW||) 

= (l-A'^(i?(0,e)))9.(e/r/i/^), 

so that 

^ (1 - A<^(5(0, £))) ^(e/r^V'^) — > (1 - A''(B(0, e))) sup (/.(t). 

»?-l-0 t 5. 

Since g{r\) ^ supj > q and £ > was arbitrary, the assertion follows. □ 



3 The upper bound (1st step) 

In this section, we consider an original X with law [i <C A''. Moreover, we assume that /x 
is compactly supported and fix Z > large enough so that /^(C) = 1 for C = [— Z, lY . 

Based on a given integrable function ^ : R'' — )■ [0, 00) {point density), we define code- 
books and control their efficiency. 

Proposition 3.1. There exist codebooks C{N), N 1, such that limiv^oo ;^ |C(A^)| = 
ll?lli,i(Md) and 

limsupE(^(ArV'i(^(X,C(Ar))) ^ [ g{^{x)) dn{x). 

Proof. It suffices to prove the assertion for functions ^ that are uniformly bounded away 
from on C. If this is not the case, one can consider ^ = ^ + elc for some e > 0. 
Then the statement says that there exist codebooks C^{N), N ^ I, with [^^(A^)! ~ 
Nm\\L\U'^}+e\''iC)) satisfying 

\imsupEipiN^/UiX,C'{N))) ^ [ g{i{x))dn{x) ^ [ g{i{x)) d^{x), 

and a diagonalization argument for e J, proves the general assertion. 

Fix m G N, let Ci = [0, //2'")'^ and decompose C into a finite disjoint union 

M 
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where M = 2('"+i)'^ and Cs, . . . , Cm are translates of Ci. For i = 1, . . . , M, we denote by 
Xi a uniformly distributed r.v. on Q, and let /x"* = lJ^{Ci)U{Ci). Moreover, we let 



M 



1)? 



and denote by u the measure given by v{A) = f^^dX'^, A € B{M.'^). 
We introduce the codebooks of interest. For some fixed k > 0, let 

C{N) = {k n-^/'^z'^) nc, N^l, 

and let Ci{N) denote codebooks of size iVj = Ni{N) = N i'{Ci) minimizing 
We consider the efficiency of the codebooks 

M 



CiN) = C{N) U (J Ci{N), N^l. 



1=1 



First, note that 



hence: 



|C(Ar)| < {i + 2-n^/^Y^(-Yn, at. 



CX), 



|C(iV)|< (||aLi(M<^)+(^)')iV, N^oo 



(10) 



It remains to estimate the expectation Kip{N^/'^d{X,C{N))) for large iV ^ 1. Observe 
that d{x,C{N)) ^ kN~^/'^ ^^Px^lo,!)'' Ikll all a; G C so that 



(11) 



E^{N^/'^diX,C{N))) - j ip{N^/'^d{x,C{N)))di/^{x) 

X,C{N))) {h{x) — hm{x)) dx\ ^ ip{cK) \\h — hm\\L'>-{Rd), 

where c = sup2,gjo,i)d II^^H is a universal constant. Moreover, 

M 

ip{N^/'^d{x,C{N))d^i"''{x) = ^n{Ci)Eip{N^/'^ d{Xi,C{N))) 

i=l 

M 

^ ^fi{Ci)Eif{N'/''d{Xi,Ci{N))). 



Now let U denote a W([0, l)'^)-distributed r.v. Due to the optimality assumption on the 
choice of Ci{N), a translation and scaling then yields 



E 



<f{N'/'' d{Xi, Ci{N))) =E^ (^'/'^ \\U- U^""'^ ll) , 
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where U^^'^'f denotes a reconstruction minimizing the latter expectation among ah r.v. 
with a range of size Ni. Next, rewriting the previous expectation as 

it follows that 

M 
i=l 

As N ^ oo, every Ni, i = 1, . . . , M, converges to oo, and one has 

where = J2i=i \d^c\ ' • Putting everything together yields 

limsupE(^(iV^/'^d(X,C(iV))) < g{^rn{x)) dn{x) + K\\h - hmWL^R'') sup 

The function converges to ^ as m ^ oo in A -a. a. points x fsee lCohnI (jl98nl ). Theorem 
6.2.3). Recall that by construction is bounded from below on C, and hence dominated 
convergence gives 

lim / g{^rn{x)) d^ix) = / g{S,{x)) dn{x). 



Analogously, hm converges to h in A'^-a.a. points x and due to Scheffe's theorem (see 
Bilhngslevl ^197% . Theorem 16.11) h m converges to h in L (M. ) as ttt, — > oo. 



For arbitrary e > 0, we can choose k > sufficiently large to ensure that the size of 
C{N) (see (HUD) satisfies 

\cm<a+e)m\LHR-^)N. 



Finally, it remains to pick m € N sufficiently large so that 

limsupE ^{N^^"^ d{X,C{N))) ^ / g{^{x)) dfi{x) + e, 
and the general statement then follows from a diagonalization argument. □ 



4 The lower bound 

From now on, let X be an arbitrary random vector on with law /x. In this section, we 
change our viewpoint: for an index set I C [l,oo) with sup I = oo, we consider arbitrary 
finite codebooks C{N), € I, and ask for asymptotic lower bounds of 

Eip{N^/'^ d{X,C{N))) 
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as — )• oo. Our computations are based on the assumption that the empirical measures 

xeCiN) 

associated to C{N) converge vaguely to some locally finite measure v on W^. 
Proposition 4.1. Letting Vc denote the absolutely continuous part of u, one has 

lirmnfE99(iVi/^d(X,C(iV))) ^ j g{^) dfi,{x). 

Proof. It suffices to prove that for an arbitrary / > 0: 

limmm^{N^/''d{X,C{N))) ^ [ g{^) dfi^ix). 

N^oo J [-1,1)'' 

Indeed, the assertion then follows immediately by monotone convergence. For a given 
m € N, just as in the proof of the upper bound, we decompose the set C = [—1,1)'^ into 
a disjoint union C = [jfiiCi, where M = 2™+\ Ci = [0,//2™)'^, and C2,...,Cm are 
translates of Ci. Again we consider the measure ^"^ = J2i=i ^^c{Ci)U{Ci) and the density 

Analogously, we let = ^^=1 \d^c.) ' ^Ci ■ For some fixed k > 0, we extend the codebooks 
C{N) to 

c^^\n) = c{N) u {{kN-^/'^z^) n c). 



Then, just as in pT|) . one has 



c JC 



^{N^'U{x,e^^\N)))diic{x) - l^v{N'/''d{x,C^'HN)))dfi"'{x) ^ ^{ck) \\h - h^h^c), 

(12) 



where c = sup^g[Q^]^)d 

Next, we control the approximation efficiency of C^^\N) for the measure /x™". We fix 
e € (0, //2™"''^) and consider for i = 1, . . . , M, the closed cubes 

a = {xeR'' ■.d2{x,{Ciy)^e} cCi. 

Here d2 denotes the standard Euclidean metric on M.'^. Now observe that there exists a 
finite set /C = IC{e) C R'^ such that for x G Cf , z = 1, . . . , M, 

d{x,ICna) i^d{x,{Ciy). (13) 

We extend the codebooks C^^\N) to C(2)(iV) = C(i)(iV) U /C and let C^{N) = C^^\N)r\Cf 
for G I and i = 1, . . . , M. Note that property (jl3p guarantees that any point x in an 
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arbitrary cube Cf has as best C^^-'(A^)-approximant an element in Ci{N). Moreover, none 
of the codebooks Ci{N) is empty, i.e. the number Ni = Ni{N) defined as Ni = \Ci{N)\, 
is greater or equal to 1. Consequently, letting Xi denote Z//(Cf )-distributed r.v.'s, one 
obtains 

I ip{N^''^d{x,e^^\N)))d^r{x)^ [ if{N^/^d{x,C^^\N)))dfi'^{x) 

M (14) 

= riCD E v{N'/''d{X,,C^{N))). 
1=1 

Let U he a W([0, 1) '^-distributed r.v., and fix an arbitrary i G {1, . . . ,M}. Note that the 
cube Cf has side length 2~"^l — 2e, so that a shifting and rescaling yields 

Eip{N'^/'^d{Xi,Ci{N))) ^ E (/p(iVi/'^ (2-™/ - 2e) \\U - U'-^^^\\) 

= E^{{NX'^{C[)y/'^ \\U - [/(^^)||), 

where f/^^*) denotes an optimal approximation satisfying the range constraint [range (U^^^^)\ ^ N^. 
With fNiv) as in ([5]), we arrive at 

E(^(iVi/'id(X„C,(iV))) ^ fNAN^/{NX''{Cm■ 

We need to control the quantity Ni/N for N large. Recall that C^'^\N) is the union of 
the sets C{N), IC and (kA^'^/'^Z'^) n C, and the vague convergence of to u implies that 

\C(N)nCi\ 

limsup^4- ^ ^ iy{Ci). 

Moreover, the set {kN^'^/'^I/) Pi Cj contains at most + l)*^ elements, so that 

N 

Consequently, Theorem 12.11 implies that 



lim sup— ^i/(Ci) + 



E^{N'/''d{XMN))) > g{{m) + /x'^iCf)). 



id 

K' 

Combining this estimate with (jl2p and ()14p yields 

ip{N^IU{x,C))diic{x) ^ [ ^{N'^''d{x,C^'^))dfi'^{x)-ip{cK)\\h-h^\\mc) 
c Jc 



i=l 



ip{cK) \\h - /imllLi(C) 
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as — )• oo. Since e > can be chosen arbitrarily small, it follows that 

M 



{im{x) + ^ dflc{x) - ip{cK) \\h - hmWLHO 



i=l 

c 



As m ^ oo, the densities converge pointwise to ^ = ^ for A'^-a.a. x, and hm converges 
to h in L^{C). Consequently, Fatou's Lemma implies that 

liminf / 99(Afi/'^d(x,C(2)))d^e^ / gU{x) + ^) dfi,{x). 
^^oo Jc Jc ^ >^ ' 

Finally, observing that k > was arbitrary and applying monotone convergence yields the 
general result. □ 

The above proposition enables us to give a partial proof of Theorem 11.31 For the 
remainder of this section, let / be given by ([3]), assume that / G (0,oo), and denote by 
M. the set of measures associated with the minimizers of the point allocation problem as 
defined in (jH). So far we have not proved that M is non-empty. 

Proposition 4.2. Suppose that the codehooks C{N), G N, are of size N and satisfy 

limsupiV^/'^ ||d(X,C(iV))||^ ^ /^/^ (15) 



and consider the associated empirical measures 

1 



Then {h'^)NeN is a tight sequence of probability measures and any accumulation point of 
{v^) lies in A4 (in the weak topology). 

Proof. Fix an arbitrary vaguely convergent subsequence {ly'^) ngi of (i^'^) NeN and denote 
by v its limiting measure. Let e > 0. As long as the Orlicz norm C(A^))||,^ is finite, 

one has in general 

Note that ()15p implies that for all sufficiently large NeN 

\\diX,C{N))\\^^{{l + e)I/N)'/'' 

so that 

limsupE(^((iV/(l +e)/)^/'^d(X,C(iV))) ^ 1. 

We consider the codebooks C{N) = C((l + e)NI) for 

iV G i := {iV/((l +e)/) -.N el}. 
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Then 



limsupEip{N^/'^d{X,C{N))) ^1. (16) 

JV-5>oo 

On the other hand, the empirical measures 

converge vaguely to (1 + e)!^ so that by Theorem 14. H 

limmiEip{N^/'^d{X,CiN))) ^ [ §{{1 + e)I C{x)) dudx), 

TV-i-oo J 

where ^ = . Combining this with (|16p , and noticing that e > is arbitrary, one obtains 

I g{n{x))d^i,{x) ^ 1. 

Consequently, the point allocation ^{x) = I ^{x) solves the allocation problem: 

j gi^ix)) dficix) ^ 1 and j idX'^ ^ /. (17) 

Due to the definition of /, the right inequality is actually an equality. 

Assume now that / g{6,{x)) dfj,c{x) < 1, and fix 5 > (small) so that the set A := 
{x G M'' : ^(x) ^ 6} has positive Lebesgue measure. Since g restricted to [5/2, oo) 
is Lipschitz continuous, we can lower the density ^ on ^ in such a way that the point 
allocation constraint remains valid, thus contradicting the optimality of /. Consequently, 
the inequalities in ()17p are even equalities, and we immediately obtain that Uc A4. Since 
fc has mass 1, we also have that u = Uc ^ A4- Moreover, {i'^)]\f^i converges to u in the 
weak topology. We finish the proof by noticing that the sequence (i^^) is tight, since it 
has no vaguely convergent subsequence loosing some of its mass. □ 



5 The point allocation problem 

We decompose the original measure /i into its absolutely continuous part /Xc = hdX'^ and 
singular component /i^. The singular component will have no influence on the asymptotics 
of the quantization error. 

In this section we use standard methods for convex optimization problems to treat the 
point allocation problem, i.e. the minimization of 

/ ^{x)dx (18) 
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over all positive integrable functions ^ : — )• [0, oo) satisfying 

5(e(x))d^c(x) ^ 1. (19) 



A minimizer will be called optimal point density. 
We shall use the convex conjugate of g, i.e. 

g* (a) = sup [oT] - g{rj)], a ^ 0, 

and the concave function g : [0, oo) [0,oo),a i-^ —g*{—a). Alternatively, one can define 
g as g{a) = inf^ ^ o[ar] + g{rj)]. 

The function g is continuous and satisfies ^(0) = inf^ ^ o div) — 0- The right conti- 
nuity in is a consequence of the lower semicontinuity of g* . Moreover, since g is lower 
semicontinuous, one has 

g{r]) = sup [ar] - g* (a)] = sup [g (a) - ar]], r] ^ 0. (20) 

a s£ a 3: 

Theorem 5.1. 1. The minimal value I satisfies the dual formula 

I = sup-( I g{--^) dfic{x) - l). 

2. The optimization problem has an integrable solution ijf the integral 



(21) 



S(^)<iA.<:(-) (22) 

is finite for some k > 0. In such a case there exists an optimal point density ^. 

3. Suppose that ^-^ finite and that 

^ic(^'^) supifit) > 1. 

t ^ 

Then I > and there exists an optimal point density. Moreover, all optimal point 
densities ^ satisfy 

g{^{x)) dficix) = 1 and g'^[-^) <^ ^{x) ^ g'_{j^) for a.e. x eR'^, (23) 

where k is a maximizer of the right hand side of 12 (Here we make use of the 
convention that g'_^{oo) = g'_{oo) = 0). In particular, the supremum in the dual 
formula is attained. 

4. ///x,(M'^)supi > fi't) ^ / = and ^ = is an optimal point density. 
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Remark 5.2. Assu me that / £ (0, oo) and that g is strictly convex. A standard result from 
convex analysis (see Rockafellail (1970), Theorem 26.3) states that the strict convexity is 



equivalent to differentiability of g. Hence, one obtains a one parameter family of candidates 
as optimal point densities. Moreover, 

g'{a)=mf{b>0:-g'4b)^a}. (24) 

Additionally, the strict convexity implies almost everywhere uniqueness, since for two 
optimal solutions .^i , ^2 that were not almost everywhere identical the combination ^ = 
|(Ci + ^2) would satisfy / g{^{x)) dx < 1 and by continuity of g it is straight forward to 
construct an admissible density with smaller L^-norm. 

If g is additionally differentiable, the one parameter family of candidates is given via 

with the convention that {—g')^^{oo) = 0. 

Proof of Theorem 15.11 By the concavity of g, the integral ()22p is either finite or 
infinite for all k > 0. We start with proving the " ^ " inequality in the dual formula. Note 
that by definition of ^, a 6 ^ g{a) — g{b) for a, 6 ^ 0. Therefore, for k > and ^ satisfying 
dlH]), it is true that 



/ (,{x)dx^ [ (,{x)dx = - I i{x) dnc{x) 

J J{h>a} 1^ J Hx) 

(25) 

In order to have equalities in the above estimates, we need to find a density ^ and 
K > such that 

^(x) = for A'' a.a. x e {h = 0}, (26) 
9{h^)-9{^ix)) = hk^ix)^ for/xea.a. X (27) 



and 



/ 



ff(e(x))d/i,(x) = l. (28) 



In the case ficO^'^) sup^ ^ q (p{t) ^ 1, it is easily seen that ^ = is an optimal point 
density so that / = which proves assertion 4. Moreover, the term on the right hand 
side of ([2T]) tends to when letting k — )• 00 so that the dual formula is valid in that case. 
Moreover, if ([5]) is infinite for one k > 0, then there is no integrable nonnegative function 
^ satisfying ()19p and the dual formula is valid as well. 

From now on, we assume that ^ci^"^) sup^ ^ q ip{t) > 1 and that is finite for any 
K > 0. Next, we derive a density ^ satisfying the three abovementioned conditions. Then 
estimate (|25p implies optimality for this choice of ^ which proves assertion 1. 
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First we examine the second condition. Consider ao,6o ^ with 

9+{ao) ^bo^ g'_{ao). (29) 

Due to the concavity of g it holds that g{a) ^ g{cLo) + bo{a — oq) for all a ^ 0, and we 
obtain with (pOD . 



5(60) = sup [g{a) - boa] ^ sup [g{ao) + 6o(a - oo) - 60 a] = 5(00) - &oao- 

a > a > 

Consequently, condition ([29]) implies that 

9{bo) = g{ao) - boat). (30) 

Conversely, it is easy to see that any pair (oq, ^o) of nonnegative reals satisfying ([5U|) also 
satisfy (12^1) . 

For K > 0, we consider the point densities 



and 



X e 



/i(x)- 

with the convention ^^^(oo) = g'_{oo) = so that, in particular, C-i^) = ^+ix) = for 
X £ {h = 0}. Furthermore, for any x G {h > 0}, condition ([JSD is satisfied for qq = K/h{x) 
and for all 69 G [C+(x), ^^!(a;)]. Therefore, every convex combination ^ = + (1 — a) 
satisfies 

In analogy to (|25p we obtain that 

J ^{x)dx = ^(^j a^j^) dficix)- j giC{x)) dncix)y 

In particular, all three integrals are finite. It remains to find an appropriate k > and a 
convex combination ^ as above with 

g{i{x)) d^ic{x) = 1. 

We need to compute the asymptotic behavior of g{g'_[a)) as a — )• and a — )• 00. Due 
to equation (pO]l . one has for all a > 0, 



Oig'-ia)) = g{a) - g'_{a) a ^ g{a), 
and we obtain that lima^o gid'-id)) = 0. On the other hand, for any 6 > 0, 

'aia) 



g{b) = sup [g{a) — ah] ^ lim sup a (- , 

a > n a->oo \ a ) 
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and, since g{b) is finite, it follows that linia-^oo g{a) /a = 0. Thus the concavity of g implies 
that lima_j>oo g'- (o) = and we arrive at 

lim g{g_{a)) = g{0). 

a— >oo 

The above asymptotics imply with monotone convergence, that 

lim / g{C{x))d^,{x) = 0, 

and 

lim / g{e{x))d^l,{x)=g{0)^i,{R'') > 1. 

Now let 

Kq 



sup{k>0: J gidx)) dficix) < l] ■ 



As we have seen above the set is non-empty and bounded so that kq G (0, oo). Moreover, 
since for any x E M'^, k i— )• £,1{x) is decreasing and left continuous, it follows by monotone 
convergence that 

[ g{e'{x))dficix) = lim [ giCix))dfi,ix) ^ 1. (32) 

J KtKO J 

Similarly the inequality ^+(2;) ^ £,1{x) implies that 

/ 5(^(2;)) d/ic(x) = lim / g{^l{x))df,Ax) 

^ lim / gidx)) dficix) > 1. 

kIkq J 



Hence, inequalities ([32]) and (j33|l imply the existence of a convex combination ^ = a + 
{l-a)^! with 

J 9{i{x)) dnc{x) = 1. 

This function ^ solves 

/ =Voij K/i)) - 1) = Kxr) - 

and we proved assertion 1. Moreover, we observe that for any ^ which is not of the 
form (|23p there is a strict inequality in at least one of the estimates in (|25p . 

□ 
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6 The singular case 

In this section, we consider an original X with law /iJ.A'^ or, equivalently, = 0. Moreover, 
we again assume that fi is compactly supported. 

Proposition 6.1. There exist codebooks C{N), N ^ 1, with lim7v->-oo j[\C{N)\ = such 
that 

lim E^{N^/'^d{X,C{N))) =0. 

N—^oo 

Proof. We fix e > 0. As the measure /x is singular with respect to the Lebesgue measure 
A''^, there ex ists an open set ^ C M'^ with fi{A) = 1 and X'^{A) ^ e. Due to Lemma 1.4.2 in 
CohnI (jl98d ^. one can represent the open set A as a countable disjoint union of half-open 



cubes (Ci)i6N in Um=i ^m, where 

Bm = {[n2-'", (ii + 1)2-'") X • • • X [^,2-"^, (id + 1)2-"^) : n, . . . , irf e Z} C M-^ 
for m G N. Due to monotone convergence, we obtain that there exists M € N with 

M 

f^{\JCi) > KA)-e = l-e. (34) 
1=1 

Set C7 = Uf=iQ. 

Let us introduce the codebooks; fixing / > such that supp ifi) C [-IJ)"^, the con- 
struction depends upon two parameters ki, ^2 > 0: 

C{N) = {{ni N-^/'^ z^) n c) u ((k2 n-'^''^ z'^) n [-/, i)'^) , N^i. 

We need to control the size of C{N). For i £ {1, . . . , M}, let rrij G N denote the unique 
number with Ci G Brm , and observe that 

, 9—'^' \ 1^ 

as ^ 00. Analogously, 

2Z 

Consequently, 

A/ 



\C{N)\ ^ J^|(KiA^"i/"'z'^)nQ| + |(K2Af-^/'^z^)n[-/,/)'^| 



■t=i 



< 



Next, we estimate the approximation error. Suppose that ^ 1 is sufficiently large so 
that Ci n C(A^) 7^ for alH = 1, . . . , M. Let c = sup^g[o,i)'* ll^ll) ^'^d observe that for all 
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X G C, d{x,C{N)) ^ CKiN-^/'^. Moreover, for any x G [-/,/)'^, d{x,C{N)) ^ CH2N-^/'^. 
Consequently, 

E^iN'/UiX,CiN))) ^ KC) vicK,) + (1 - ^(C)) ^{CK2) ^^^^ 
^ V3(cKi) + ev3(cft;2)- 

Now, for 6 > arbitrary, pick ki,K2 > satisfying (p{cK,i) ^ 6/2 and (21)'^ ^ '^/^^ 
and choose e > so that e^j"'^ ^ J/2 and £(p(cK2) ^ Then the corresponding 

codebooks C{N) satisfy 

\C{N)\<5N and E (^(iW'^ ^(x, C(iV))) < 5, 

and the assertion of the proposition follows by a diagonalization argument. □ 

7 Extension to the non-compact setting 

In order to treat the non-compact quantization problem, we need to control the im pact 
of r ealizations lying outside l arge cubes. For LP(P)-norm distortions, iPierce (|l970l ) see 



also Graf and Luschgv ( 2000l ) . Lemma 6.6) discovered that the quantization error can be 



estimated against a higher moment p > p of \\X\\. His result can be easily extended to 
the inequality 

S{N\X,p) ^ CE[\\X\\P]^/PN-'^/'^, 

where X is an arbitrary original in M'^, G N and C is a universal constant depending 
only on E, p and p. Pierce's proof is based on a random coding argument. In contrast to 
his approach, we will use e-nets to establish a similar result. 

The construction is based on several parameters. Let : [0, oo) — > [0, oo) denote 
an increasing function, {rn)neNo an increasing sequence, and let (an)neN be a positive 
decreasing and summable sequence. 

Lemma 7.1. Let J G No and denote by X a {B{0,rjy U {0}) -valued r.v. For N ^ 
there exists a codebook C{N) of size 1 + J2'?^=j satisfying 

oo ^ 

E^{N'/U{X,C{N))) ^ E[^{\\X\\)] ^ ^(cEa-'/'rn-.i), (36) 

n=J 

where ce is a finite constant depending on the norm \\ ■ \\ only. 

Proof. First, observe that the sum in estimate (j36j) diverges whenever lim inf^_^QQ ^ oo. 
Thus we can assume without loss of generality that lim„^oo''n = oo. Fix J G No and 
G N. Forn G Nq, let K = 5(0, r„) and 7V„ = a„7V. Moreover, we denote by / G No the 
smallest index n with Nn < 1, and let for n G No with J ^ n < I , Cn denote an optimal 
e-net for B{0, r^+i) consisting of A^^^ elements. As is well known there exists a constant ce 
only depending on the norm || • || such that d{x,Cn) ^ CErn+iNn ^^'^ for all x G i?(0,r„+i). 
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Thus the codebook C = {0} U IJn=j ^™ contains at most 1 + SJ^j o^n elements, and we 
have 

oo 

n=J 
oo 

n=J 
oo 

n= J 

oo ^ 

^E[vl/(||X||)] -y(^(c^r„+ia;;V^). 

r!,=J 

□ 

Definition 7.2. We say that an increasing function ^ : [0, oo) [0, oo) satisfies the 
growth condition (G) for if iff there exists a decreasing summable sequence (a„) and an 
increasing sequence (r„) with 

oo ^ 
n=0 ^ 

Suppose that ^' satisfies condition (G) for cp. We shall see that the condition that 
E^'(||X||) < oo is sufficient to conclude that the quantization problem for X under the 
Orlicz norm induced by (p is of order N~^^^. Moreover, quantization of non-compact 
measures then can be approximated by the compact setting. 

Example 7.3. • Suppose that (/?(t) ^ c(l + t^) for all t ^ 0, where c,p (z are 
appropriate constants. Then, for any f3 > 

^(t)=tP(log+t)'^ 

satisfies (G) for (p. For instance, one may choose {rn)neNo = (2"')nGNo («n)nGNo = 
((n + 2)-^)„eNo foraTG (l,^(/3-l)). 

• Suppose that ip satisfies ip{t) ^ c expjt''} for all t ^ 0, where c,kG M+ are appro- 
priate constants. Then, for any R> k, the function 

^{t) = exp{t'^} 

satisfies (G) for ip, as can be verified easily for (an)nGNo = ((^ + 2)^^)neNo and 
{rn)n&io = {{n- + l)'')neNo for s > with sk > (| + s)k. 

Remark 7.4. The proof of the upper bound in Theorem 11.11 relies on the assumption 
that E^'(||X||) < oo for some ^ satisfying the growth condition (G). As we shall see 
below, this assumption can be replaced by the equivalent condition that X € L*(P) for 
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some ^ satisfying (G). First, assume that E^'(||X||) < oo for some ^ satisfying (G). Tlien 
^ = l[i,oo) ^ satisfies (G), and since by monotone convergence 

lim K^{\\X\\/k) = 0, 

it follows that X G L"^(P). On the other hand, assuming that X € L'^(P) for some ^' 
satisfying (G) implies the existence of a k > for which 

K^{\\X\\/k) < oo. 

Now, let ^{t) = ^{t/ k), and denote by and (r„) sequences as in Definition 17.21 Then 
E^dlXll) < oo, and 



oo ^ oo ^ 



OO 



n=0 ^ n=0 

for d!„ = K'^a^ and = KTn, n S Nq. 

We now combine the quantization results for continuous, singular, and unbounded 
measures to finish the proof of the upper bound in Theorem II. li 

Proof of Theorem II. IL Let be a function satisfying (G). It is easy to see that there 
exist also a summable and decreasing sequence (a„) and an increasing sequence (r„) such 
that 

oo ^ 

Yl ^(TT '^(^^ "^^^"^ < 

n=0 " 

where ce is as in Lemma mi We denote by an optimal point density, so that ^ satisfies 

/ g{^{^)) dfJ.c{x) = 1 and / ^{x)dx = I 

or ^ = (in the case 1 = 0). We fix e > and let i{x) = ^(x) +£h{x), x G M"^. This point 
density satisfies 

/ giiix)) dfic{x) < 1 
since g is strictly decreasing on {r/ > : g{r]) > 0}. Now fix J G No such that 

oo 

/ giiix))dfi,ix)+E[^i\\X\\)] ^(cEa-'/'^rn+i) < 1 (37) 

and Yl'^=j '^n < Next, decompose the measure fi into the sum fj. = flc + fis + fJ-u, where 
jlc and jls are the absolutely continuous and singular part of /i restricted to B{0,rj), 
respectively, and contains the rest of the mass of /i. 

It remains to combine the former results. Due to Proposition 13.11 there exist codebooks 
Ci{N), iV ^ 1, with limjv^oo jr\^i{N)\ = lieilLi(Rd) and 

limsnp [ ip{N^/'^dix,Ci{N)))dfic^ I g{i{x)) dii,{x). 
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Moreover, Proposition l6.1l implies the existence of codebooks C2{N), N ^ 1, with hmiv_^oo -^|C2(A^)| 
and 

hm [ (p{N^/'^d{x,C2{N)))dfls = 0. 

N^oo J 

Finally, Lemma ITT] (applied to X = lB(^o,rj)''i^) ' ^) yields the existence of codebooks 
Cs (TV), iV ^ 1, for which 



n=J 



^ oo 

limsup — |C3(iV)| ^ ^a„<e 

and 

ip{N'/''d{x,C3{N)))dfi^ ^E[^{\\X\\)]T —^^{C a-' 

n=J 



Now consider the codebooks C[N) = Ci(N) UC2{N) UC^{N). Due to the above estimates 
and ([37|l . one has 

limsup [ip{N^/'^d{x,C{N)))dfi 



oo 

SO that for sufficiently large it is true that ||(i(X, C(A^))j|^ ^ N^^/'^. On the other hand, 

limsup 4|C(A^)| <(! + £)/ + £ 

and, for sufficiently large N , it holds that jC(A^)| ^ {I + el + e)N . Consequently, it follows 
that for large 

6{{I + el + e)N\X, if) s: N'^/'^. 
Switching from N to M = (I + el + £)N one obtains 

6{M\X,ip) ^ (/ + e/ + e)i/'^M-i/^ 
for M large. Since e > was arbitrary, it follows that 

limsupM^/'^5(M|X,(^) ^ /^/'^ 

and we proved the upper inequality. 

In order to prove the lower bound we fix codebooks C{N), N & N, with at most N 
elements and 

limsupiV^/'^5(iV|X,v9) ^ I^/'^. 
By Proposition 14. 2^ each accumulation point of the associated empirical measures 



- y 
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lies in A4. In particular, 



Therefore, for any e G (0, 1), 



hm l^M = 1, (38) 



liminfiVi/'^5((l -e)iV|X,v3) > 7^/'^. 

iV— >-CXD 

Otherwise one could construct a sequence of codebooks C{N), G N, as above which 
does not fulfil ()38p . Switching from to M = (1 — e)A^ and letting e J, we obtain the 
lower bound. 

The remaining properties of the minimizer / were proved in Theorem 15.11 □ 
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